A Hybrid Approach to Design Neural Network Ensemble

ACKNOWLEDGEMENTS We would like to express our heartiest gratitude and thanks to our advisor, Dr. Md. Monirul Islam, for his time, advice, encouragement and guidance throughout our thesis. We are very fortunate to work with him and have benefited greatly from his advice. We are very much grateful to Dr. Muhammad Masroor Ali, the Head of the department, for assuring a good atmosphere for research in the labs by providing us with internet and other facilities. We are also grateful to the lab attendants of our software engineering lab and computing lab for their heartiest cooperation. ABSTRACT This thesis presents two cooperative ensemble learning algorithms, NegBagg and NegBoost, for designing ANN ensembles. NegBagg and NegBoost train different individual ANNs in an ensemble incrementally by using the NC learning algorithm. Bagging and boosting algorithms are used in NegBagg and NegBoost, respectively, to create different training sets for different individual ANNs in an ensemble. The idea behind using NC learning in conjunction with bagging and boosting algorithms is to make interaction and cooperation among individual ANNs in an ensemble work better. Both NegBagg and NegBoost use a constructive approach to determine the ensemble architecture automatically. They have been tested on a number of benchmark problems in machine learning and ANNs, including Australian credit card assessment, soybean and waveform. The experimental results show that NegBagg and NegBoost can produce ANN ensembles with good generalization ability by using a small number of training epochs.